Security  Classification _ _ 


DOCUMENT  CONTROL  DATA  -RAD 

_  (Security  ctaaaification  of  title,  body  ci  abstract  an d  indexing  mnnotmtion  nu*i  be  entered  when  the  o varatl  report  u  ctaas Hied 


1.  ORIGINATING  ACTIVITY  (Corporate  author)  I  2*.  REPORT  SECURITY  CLASSIFICATION 


North  Carolina  State  University 
Department  of  Electrical  Engineering 


UNCLASSIFIED 


CROUP 


3,  REPORT  TITLE 

DATA  COMPRESSION  PANEL  DISCUSSION 

_ _ * 

4.  DESCRIPTIVE  NOTES  (Type  n(  report  and  inclusive  deter) 

Scientific  Interim 

Is.  AUTHORISE  (Firet  new*,  middle  i nitial,  last  name) 

Jay  W.  Schwartz  Thomas  S,  Huang 

Jacob  Ziv 

Lee  D.  Davisson  J.  B.  O'Neal,  Jr. 

6.  REPORT  DATE 

7a.  TOTAL  NO.  OF  PAGES 

76.  NO.  OF  REFS 

4  April  1972 

5 

29 

8a.  CONTRACT  OR  GRANT  NO. 

9a.  ORIGINATOR'S  REPORT  NUMBER(S) 

F44620-69-C-0033 

6.  PROJECT  NO. 

7921 

C-61102F 

f  9b.  OTHER  REPORT  NO(S)  (Any  other  number*  that  may  be  assigned 

fhl<  report) 

■  ~  j 

/W 

*681304 

AFOStt  -  TR  -  V 

10,  OiSTRIBUT.ON  STATEMENT 

Approved  for  public  release;  distribution  unlimited, 


11.  SUPPLEMENTARY  NOTES 


TECH,  OTHER 


i^TRACT 


12  SPONSORING  MILITARY  ACTIVITY 

Air  Force  Office  of  Scientifix?  Research 
1400  Wilson  Boulevard  (NM) 

Arlington,  Virginia  22209 _ 


Data  compression,  in  fch4:6  paper,  is  used  to  denote  the  reducing  of  an  input 
data  set  prior  to  transmission,  as  opposed  to  data  reduction,  the  analytical  pro¬ 
cessing  of  the  set  upon  reception.  Source  encoding  techniques  are  classified  into 
sampling  and  analog  to  digital  conversion,  codebook  techniques,  predictive  sub¬ 
tractive  ceding  and  delta  modulation,  along  with  aperature  and  partitioning 
techniques^  Some  of  the  most  recent  work  discussed  includes  adaptive  predictive 
coding,  picture  bandwidth  compression-source  coding,  and  a  techniques  for  subjective 
performance  measures.  The  paper  concludes  with  an  information  theoretic  approach 
to  data  compression. 

national  technical 
information  service 

Springfield  Va  22151 


DD  ,'““,,1473 


Sfj-urity  Classification 


AD  ^4  0(5 1 5 


DATA  COMPRESSION  PkMEL  DISCUSSION 

Jay  W.  Schwartz,  Chairman 

Institute  for  Defense  Analyses 
Arlington,  Virginia  22202 

Lee  D.  Davisson*  *  J.  B.  O’Neal,  Jr, 


University  of  Southern  California 
Los  Angeles,  California  90007 

Thomas  S.  Huang 

Massachusetts  Institute  of  Technology 
Cambridge,  Massachusetts 

I.  INTRODUCTION 

The  term  data  compression  is  used  to  de¬ 
note  operations  whTch  are  performed  to  reduce 
data  prior  to  transmission.  This  technology 
has  also  been  called  by  other  names:  "data 
compact  ion* ,  "bandwidth  compression",  etc. 

The  term  data  reduction  usually  refers  to  oper¬ 
ations  performed  at  a  processing  center  to 
extract  information  in  a  convenient  form  for 
interpretation.  Data  compression  results  when 
data  reduction  is  performed  at  the  transmitting 
end. 

It  is  convenient  to  differentiate  bptween 
two  forms  of  data  compression:  extraction  and 
source  encoding.  Extraction  is  a  term  for 
data  reduction  performed  at  the  transmitter 
rather  than  the  receiver  in  order  to  reduce 
the  amount  of  data  transmitted.'  It  refers  to 
operations  which  "extract"  desired  information, 
casting  aside  unimportant  or  less  important 
properties  of  the  data..  Examples  of  extraction 
opeutions  performed  in  space  programs  are  the 
calculation  of  moments,  discarding  samples 
with  values  below  a  threshold,  and  discarding 
samples  which  do  not  represent  a  local  maximum. 
Pattern  recognition  performed  prior  to  trans¬ 
mission  is  a  form  of  e  fraction.  In  contrast 
to  source  encoding,  ex  -tion  operations 
normally  would  be  pert'  _d  at  the  receiver  as 
data  reduction  if  they  were  not  perfoi med  at 
the  transmitter.  Source  encoding  refers  to 
operations  which  produce  outputs  (encoded  data) 
from  which  the  input  data  can  be  reconstructed, 
possibly  with  distortion.  To  obtain  data  com¬ 
pression,  source  encoding  relies  on  knowledge 
of  source  statistics  or  on  tolerance  for  dis¬ 
tortion  at  the  receiving  end.  Unlike  extrac¬ 
tion  operations,  source  encoding  operations 
usually  would  not  be  considered  as  a  step  in 
data  reduction;  indeed  the  first  step  in  data 
reduction  at  tne  receiver  probably  is  to  urtdo 
them:  i.e.  ,  to  decode.  Extraction  in  general 
has  potential  for  much  greater  compression  of 
data  than  does  source  encoding.;  However  ,  ex¬ 
traction  techniques  often  are  outside  tne 
interests  of  communication  ♦•heory.  Therefore 
this  discussion  will  be  confined  to  source 
encoding. 

Shannon1  has  considered  the  problem  of 
coding  an  information  source  provided  that 
there  is  a  way  to  measure  distortion  of  th( 
final  recovered  Information  at  the  receiving 
point.  He  showed  that  a  function  R(D), 
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called  the  rate  distortion  function,  exists 
which  is  a  measure  of  the  information  r^te  re¬ 
quired  to  specify  the  output  of  a  souice  with 
a  distortion  less  than  D.  That  is  R(Ct,  in 
bits  per  source  symbol,  is  the  minimum  data 
rate  which  can  ae  used  to  describe  the  infor¬ 
mation  with  a  distortion  less  than  D.  This 
concept  can  form  the  basis  of  a  quantitative 
approach  to  source  encoding  in  the  following 
manner.  A  coding  scheme  would  >e  evaluated 
for  use  with  a  source  by  evaluating  data  rate 
as  a  function  of  the  distortion  measured  after 
encoding  and  decoding.  Plots  for  various 
cooing  techniques  would  be  compared  with  each 
other,  ana  with  R(D)  for  that  source.  To 
select  the  "best"  scheme  for  a  particular  ap¬ 
plication,  one  would  specify  a  maximum  distor¬ 
tion,  D*,  and  compare  performance  of  the 
various  coding  schemes  under  the  constraint 
that  D*D*.  Further,  one  would  compare  this 
performance  with  R(D* )  to  determine  how  much 
better  one  might  do  with  better  schemes.  The 
primary  difficulty  in  realizing  this  systematic 
approach  is  in  finding  suitable  distortion 
measures,  but  another  difficulty  is  that  to 
make  the  problem  tractable,  it  must  be  possi¬ 
ble  to  divide  a  message  into  independent  por¬ 
tions.  Neither  difficulty  is  easily  overcome. 
For  example,  a  picture  seldom  can  be  divided 
into  independent  segments.  Aside  from  a  few 
special  cases,  investigations  in  conjunction 
with  the  rate  distortion  function  assume  a 
mean- square -error  distortion  measure  on  a 
sample -by -sample  basis.1 » 3 »4 

This  is  not  always  an  adequate  measure  of 
distortion.  Lacking  ways  to  measure  distor¬ 
tion  quantitatively  as  would  be  recui-'ed  for 
the  approach  outlined  above,  subjr  ,ive  mea¬ 
sures  are  used  by  each  researcher  to  determine 
which  reconstructed  pictures  are  "adequate" 
and  which  are  "better"  than  others..  Similarly, 
for  voice  messages  "articulation"  or  "intelli¬ 
gibility"  is  often  used  to  evaluate  recon¬ 
structed  sound 

II. CLASSIFICATION  OF  SOURCE  ENCODING  TFCHNIQUES 

Sampling  and  A/D  Conversion  Sampling  can 
provide  data  compression,  but  usually  only 
when  the  desired  information  has  a  very  low 
bandwidth  compared  to  the  available  channel: 
for  example,  a  temperature  that  can  be  ade¬ 
quately  described  by  one  sample  an  hour.  When 
it  is  necessary  to  represent  an  analog  func¬ 
tion  such  as  a  TV  scan  or  a  voice  message  by 
PCM,  it  is  usually  found  that  sampling 
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necessitates  greater  communication  capacity 
than  the  original  analog  message.  Certain  im¬ 
plement  a  lions  of  "aperture"  techniques,  dis¬ 
cussed  presently,  may  be  considered  as  •: 

employing  asynchronous  sampling  to  achieve 
data  compression. 

Source  coding  techniques  for  data  com¬ 
pression  usually  operate  on  digital  data;  some 
because  they  must,  others  because  they  can  be 
instrumented  more  conveniently  for  digital 
than  for  analog  data.  Therefore,  when  evalu¬ 
ating  a  source  encoding  technique,  it  is  some¬ 
times  necessary  to  penalize  its  performance 
with  the  cost,  in  terms  of  bandwidth  and/cr 
signal  power,  of  analog  to  digital  convers ion. 
Moreover,  sampling  rate*  quantization,  and 
reconstruction  procedure  often  can  affect  the 
relative  performance  of  source  encoding  tech¬ 
niques.  In  some  cases  investigators  are  not 
free  to  experiment  with  these  parameters  as 
freely  as  they  should:  for  example,  TIROS 
pictures  which  have  been  used  in  evaluating 
techniques  often  have  been  sampled  and  recon¬ 
structed  before  being  made  available* 

Code -Book  Techniques  Certain  source 
encoding  techniques  can  be  considered  to  repre¬ 
sent  each  source  word  with  a  code  word.  Some 
familiar  examples  of  the  techniques  which  fall 
in  this  class  rely  on  the  statistics  of  the 
source  symbols:  the  Shannon- F* no*  ,  Huffman®, 
and  Morse  code  .  Gilbert  and  Moore7  discuss  a 
number  of  the  e  codes.  Other  examples  rely  on 
an  understand  ng  of  tolerable  distortions 
rather  than  source  statistics:  for  example, 
sophisticated  quantization  schemes  and 
companding.® 

A  few  of  these  code-book  techniques  have 
found  v/ide  practical  usage:  the  Morse  Code 
and  logarithmic  companding  are  probably  the 
best  examples.  To  take  advantage  of  correla¬ 
tion  from  one  source  word  to  another  conve¬ 
niently,  code  book  techniques  may  be  used  in 
conjunction  with  other  forms  of  source  encod¬ 
ing. 

Predictive  Subtractive  Coding  an^  Delta 
Modulation  Anotu*r  general  approach  to  source 
encoding  is  to  predict  a  sample  value,  based 
on  previous  samples,  and  then  to  transmit  the 
difference  between  true  and  predicted 
sample  values.10  Of  course,  some  form  of  code¬ 
book  technique  must  be  employed  on  the  differ¬ 
ence  values  in  order  to  obtain  data  compression. 
The  most  oraetical  implementation  of  predictive- 
subtractive  coding  is  delta-PCM  in  which  the 
difference  values  are  merely  quantized .l  1 » 1 9 
Delta-modulation  is  the  special  case  of  delta- 
PCM  with  one-bit  quantization. 

When  compared  with  most  other  source 
encoding  techniques,  delta-PCM  has  the  impor¬ 
tant  advantage  of  producing  encoded  data  at  a 
fixed  synchronous  rate*:  It  has  the  undesirable 
property  of  predict ive- subtractive  coding  that 
transmission  error  propagates  through  succeed¬ 
ing  deco/ad  values..  The  potential  compression 
available  with  delta-PCM  is  limited  to  at  best 
a  factor  equat  to  the  number  of  bits  per 
sample  in  an  eGvquate  PCM  representation, 
unloss  one  resorts  :o  supplementary  coding 
whi  :h  would  sacrifice  the  fixed  synchronous 
data  rate. 

Aperture  Techniques  Aperture  techniques 
are  similar  to  predictive-subtractive  coding 
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in  that  use  is  made  of  the  difference  between 
the  true  value  and  an  estimated  value.  With 
aperture  techniques,  however, either  the  true  or 
estimated  value  is  transmitted  rather  than  the 
difference  between  them.1** 14  Data  compression 
is  obtained  by  transmitting  nothing  when 
the  true  and  estimated  values  differ  by  less 
than  a  specified  amount,  i.e.,  when  the  true 
value  falls  within  a  certain  "aperture"  centered 
on  the  estimated  value. 

Aperture  techniques  can  avoid  the  problem 
of  error  propagation  that  results  when  differ¬ 
ences  are  transmitted.  Furthermore,  they  can 
permit  the  use  of  interpolation  instead  of  pre¬ 
diction  in  forming  estimates.  Encoded  data  is 
produced  at  an  asynchronous  rate  which  depends 
on  the  success  in  estimating  sample  values; 
because  of  this,  each  transmitted  value  must  be 
tagged  to  identify  the  sample  which  it  repre¬ 
sents. 

Partitioning  Techniques  Another  approach 
to  source  encoding  is  to  consider  the  source 
data  as  a  combination  of  distinct  parts,  and  to 
treat  each  part  separately.  The  best  known 
partitioning  technique  is  vocoding  in  which 
voice  messages  are  divided  into  component  fre¬ 
quencies  which  are  described  by  th<  transmitted 
<Jnra.  The  original  voice  message  is  recon¬ 
structed,  with  a  loss  in  fidelity,  from  the 
component  frequencies.  Somewhat  similar  philos- 
opliy  gives  rise  to  techniques  proposed  for 
encoding  television  data  by  treating  high  and 
low  frequency  components  separately,  or  by 
treating  different  bits  of  the  sample  values 
separately.1®  »lft 

For  example,  if  the  intensity  at  each 
picture  element  is  read  to  6-bit  accuracy,  each 
sample  value  can  be  treated  as  two  values--say, 
the  higher-order  3-bits  forming  one  value  and 
the  Lower-order  3-bits  forming  a  second  value. 

The  rationale  behind  this  approach  is  that  the 
higher-order  (most  significant)  bits  change 
more  slowly,  corresponding  to  lower  frequency 
components,  than  do  the  lower-order  bits. 

Carrying  this  approach  further,  bit-plane 
encoding1 *  treats  each  bit  of  a  sample  sepa¬ 
rately,  treating  a  source  producing  n-bit 
samples  as  n  binary  sources. 

III.  SOME  RECENT  WCRK  IN  SOURCE  ENCODING 

Adaptive  Predictive  Coding*  (L.D.  Davisson) 

As  noted,  compression  can  be  attained  by 
encoding  the  differences  between  the  true  and 
predicted  sample  values  where  the  prediction  is 
based  upc a  the  past  as  seen  at  the  receivev'. 

The  coding  is  done  relative  to  some  distortion 
measure--usually  it  is  required  that  the  recon¬ 
structed  sample  values  disagree  from  the  true 
values  by  less  than  a  preset  amount. 

Ideally-  such  a  predictor  would  minimize 
the  message  entropy.  As  it  is  not  known  how  to 
do  this,  one  must  resort  to  simpler  quantities, 
usually  mean  square  error  minimization1*  but 
also  possibly  conditional  probability  mode  maxi¬ 
mization.1*  The  prediction  can  in  principle  be 
nonii/  ear  as  well  as  linear,  employing  poly¬ 
nomials  in  the  sample  values1**  *  or  conditional 
histograms.80  In  many  cases  linear  methods  are 
sufficient80  whereas  in  some  cases  profitable 
use  can  be  made  of  nonlinear  methods.81 

Th  *  optimization  of  the  predictor  can  be 
done  in  advance  if  the  data  distributions  arc 
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known  or  if  appropriate  quantities  can  be  esti¬ 
mated  from  "  represent  at  ive"  samples  under  the 
assumption  of  stationarity.  In  most  applica¬ 
tion*  such  information  will  be  only  roughly 
available  necessitating  the  use  of  adaptive 
methods  if  nearly  optimum  performance  is  to  be 
attained. 

One  simple  adaptive  predictor*0**1  of  the 
form  suggested  by  Widro%r  and  Lucky**  for  dif¬ 
ferent  applications  depends  on  local  gradient 
following  and  provides  good  performance 
be inn  quite  practical  to  implement.  Suppose 
that  the  data  are  a  time  series  (»n]  with 

teceived  values  (in  the  absence  cf  channel 
errors)  (vn)*  The  n**  predicted  value  Is  a 

linear  weighting  of  the  immediate  past  M  value b: 

M 


The  prediction  error  is  then 

M 


s  -VI  any 

n  *  ■■  -i7n-n 


e  =  S  -S  =  S  * _ _ 

n  n  n  n  4—  1 n'3 


The  weight  vector  (a^)  is  adjusted  after  each 
prediction  by  the  following  algorithm 


a1?  +  ye  y 
j  T  n  yn-g 


j  = 


where  y  is  a  small  positive  constant.  Still 
simpler  versions  result  by  replacing  en*  yn_4 

or  both  by  their  signs.  It  can  be  shown  that 
the  mean  square  prediction  error  on  a 
stationary  time  series  is 

o2[M;y]  <  o?[M](1-»MyX)  4  o(y) 


where  o2[m]  is  the  minimum  variance  using  the 

2 

optimum  predictor  and  X  -  yn*  It  is  seen  that 

nearly  optimum  performance  is  attained  by  the 
adaptive  predictor  while  retaining  simplicity 
and  ability  to  follow  data  nonstationarities. 

Picture  Bandwidth  Compression-Source 
Coding 

(T*. S.  Huang) 

A  symposium  on  picture  bandwidth  compres¬ 
sion  was  held  recently  at  the  Massachusetts 
Institute  of  Technology.  It  was  the  first 
symposium  devoted  entirely  to  the  field,  and 
was  intended  to  present  th*  current  st;atv  of 
televi  .ion  bandwidth  compression  and  of  related 
areas.  The  large  and  enthusiastic  rt  sp  use  to 
this  meeting  indicates  the  growing  interest  in 
the  area. 

Many  picture  bandwidth  compression  schemes 
have  been  proposed  and  tried  out.  Mo^t  of 
these  uperate  on  intra-line  redundancy  and 
achieve  bit  rates  of  around  3  bits /sample* 
(Direct  PCM  transmission  of  pictures  requires 
7  or  8  bits  per  sample. )  One  can  attain  lower 
bit  rates  by  operating  on  two-dimensional 
redundancy.  The  moat  recent  work  in  this 


direction  ha*  done  by  mapping  portions  of 
a  picture  wit..  linear  transformation*  and  by 

quantizing  tV  inns  formed  picture  through  some 
optimum  strat*  a  procedure  that  Fourier 

transforms  bio.  s  of  a  picture  and  codes  the 
transformed  -  j  with  an  adaptive  procedure 
proaises  to  vee  qood  pictures  with  about 
one  bit  per  -  *r,tple.a 

In  tra*  flitting  motion  pictures,  one  can 
exploit  the  t  r-ce- to- frame  redundancy..  £ecau 
cf  Urn  torn  able  experimental  problems,  how¬ 
ever,  very  kittle  work  has  beers  done  in  this 
area  untij  recently.* 7 >"*  Similar  reasons  ac¬ 
count  for  vte  scarcity  in  studies  t?n  color 
picture  csi "g.  One  such  study**  suggests  that 
a  color  f  ure  can  be  sent  at  about  the  same 
bit  rate  aw  eauivalent-quality  monochrome 
picture. 

Ixx  ,dhg  ii.to  the  future,  we  expect  to  see 
experimental  j todies  on  motion  and  color  pic¬ 
ture  cc  mg  play  a  growing  role.  On  the  theo¬ 
retics  j  s\de,  it  is  hard  for  us  to  see  any 
significant  progress.  Although  Shannon's  rate 
disto  tion  theory  seems  to  provide  an  ideal 
mathematical  background  for  studying  picture 
bandwnV*.  compression,  the  possibility  of 
quantifying  picture  quality  so  that  we  may  have 
a  suitable  fidelitv  criterion  is  as  remote  as 
ever<  Let  us  hope  that*  studies  in  visual  pat¬ 
tern  recognition  may  throw  some  light  on  this 
knotty  problem. 

Bounds  on  Subjective  Performance  Measures 

"(J.  B.  01  Nea  1  ,  Jr".  ) 

Quantizing  noise  is  present  whenever  ana¬ 
log  information  is  encoded  into  digital  form 
suitable  as  an  input  to  any  digital  system  such 
as  a  computer  or  digital  transmission  line. 
Bounding  the  impairments  caused  by  this  noise 
is  one  of  the  ubiquitous  activities  of  those 
involved  in  information  theory.  The  perform¬ 
ance  metric  to  be  bounded  is  of  primary  impor¬ 
tance.  To  be  useful  it  must  measure  the 
quality  of  the  system  in  the  eyes  of  its  users. 
Bounds  on  the  ratio  or  signal  power  to  quantiz¬ 
ing  noise  power  have  been  derived.  For  speech 
and  television  signals,  however,  it  has  long 
been  known  that  noise  power  alone  is  not  a 
definitive  measure  of  signal  quality.  For 
these  signals,  noise  at  low  frequencies  is  more 
damaging  than  noise  at  higher  frequencies. 
Frequency  weighting  networks  have  been  designed 
from  subjective  tests  which  can  be  used  to 
accurately  measure  the  detrimental  effect  cf 
noise.  The  attenuation  of  these  networks  falls 
off  with  frequency  in  such  a  way  that,  when 
noise  is  passed  though  the  network,  the  output 
power  is  an  accurate  measure  of  the  subjective 
effect  of  the  noise..  The  ratio  of  signal  power 
to  frequency  weighted  noise  powe  r  is  a  much 
better  performance  metric  than  the  usual  signal 
to  noise  ratio.  Bounds  have  been  derived  for 
S/Ny,  the  ratio  of  signal  power  to  frequency 

weighted  noise  power  for  speech  and  television 
signals.  Values  of  S/Ny  greater  than  the 

bound  are  not  possible  for  any  encoding  system. 

When  expressed  in  dB  the  bounds  on  signal 
power  to  frequency  weighted  noise  power  are 
the  sum  of  three  terms:  S/Ny  *  T&  4  Tp  +  Tg. 

TQ  is  a  term  which  increases  linearly  with  bit 

D 


rate.  Tp  is  determined  by  the  statistical 

nature  of  the  signal.  It  is  a  measure  of  the 
predictability  (or  redundancy)  of  t^e  signal. 

Tg  is  a  quantity  determined  by  the  appropriate 

subjective  frequency  weighting  function.  The 
terms  T^f  Tp  and  Tg  and  the  effects  measured  fcy 

each  one  are  relatively  independent  of  the 
others,  i.e.,.  is  dependent  only  on  the  bit 

Tp  ia  determined  only  by  the  predicts- 
bility  of  the  signal  and  Tg  is  determined  only 

by  subjective  considerations  (as  embodied  in 
the  frequency  weighting  function). 

In  order  to  obtain  numerical  results  for 
bounds  of  this  type,  values  of  Tp  must  be  esti¬ 
mated  for  practical  signals  like*speech,  tele¬ 
vision  and  still  pictures.  This  is  done  at 
some  risk  for  Tp  depends  on  the  statistics  of 

the  class  of  signals  for  which  the  systems  are 
to  be  used.  The  best  we  can  do.  is  to  estimate 
Tp  for,  what  we  believe  at  this  time  to  be, 

"typical"  signals  and  calculate  bounds  baaed  on 
these  "typical"  signals.  The  calculation  of  Tg 

can  be  made  for  speech  using  the  C  message 
weighting  characteristic  and  lor  television  by 
using  the  standard  frequency  weighting  func¬ 
tions  of  Barstow  ana  Christopher. 


(J.  Ziv) 

Suppose  that  we  have  a  data  source  which 


emits  a  sequence  of  symbols  xi**2,x3  €  X  (an 
arbitrary  set)  at  a  rate  of  pg  per  second. 

This  sequence  is  fed  into  an  "encoder"  which 
assigns  to  each  successive  block  of  n  source 
symbols,  say  x  =  (x^jX^ »•  •  •  ,xf|) ,  a  channel  in¬ 
put  of  duration  of  n/pg  =  T  seconds.  At  the 

receiving  end  of  the  channel,  the  T-second  out- 
put  is  transformed  by  a  "decoder"  into  an 
n-sequence,  'ay  £  =  (xpXj  4 .  • .  ,$n)>  which  is 

delivered  to  uhe  destination.  The  "distortion" 
between  the  source  output  sequence  £  and  tha 

received  sequence  x  is  defined  as 

=  n_1  y 

k=l 

where  d(x,x)  *  0  is  an  arbitrary  function. 

The  following  distortion  functions  are 
considered? 

(a)  d(x,x)  =  lx-x| 3  (s  >  0) 

when  x  is  a  subset  of  the  reals. 

.  f°  ’  *  =  * 

(b)  d (x ,x )  =  < 

ti  ,  x  y  x 

when  X  is  a  discrete  (i.e., 
countaDle)  set. 

i 

\ 


0  ;  Ix-xl  <0 

1  ;  |x-xl  >  o 
when  x  is  a  bounded  set  typically  the 

interval 

The  classical  proMe*  is  thit  of  a 
^memoryless"  source,  where  successive  source 
outputs  are  statistically  independent  with 
identical  probability  distribution.  In  this 
case  it  is  meaningful  to  let  the  system  perfor¬ 
mance  criterion  (fidelity  criterion)  be  the 
statistical  expectation  of  the  distortion 

d^*(xfx).  A  quantity  of  interest  is  d*(T), 

the  smallest  attainable  value  of  the  fidelity 
criterion  when  the  coding  delay  is  T  seconds. 
The  following  topic*.  of  interest: 

A)  The  possibility  of  achieving  d*(T)  by  an  A/D 
converter  followed  by  an  optimal  digital 
encoder. 

B)  The  possibility  of  using  a  uniform  quantizer 
as  the  A/D  converter. 
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